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School District Leadership that Works 

The Effect of Superintendent Leadership on Student Achievement 

A Working Paper 

J. Timothy Waters, Ed.D. & Robert J. Marzano, Ph.D. 



Executive Summary 

T o determine the influence of district superintendents on student achievement and the 
characteristics of effective superintendents, Mid-continent Research for Education and 
Learning (McREL), a Denver-based education research organization, conducted a meta- 
analysis of research — a sophisticated research technique that combines data from separate 
studies into a single sample of research — on the influence of school district leaders on 
student performance. 

This study is the latest in a series of meta-analyses that McREL has conducted over the past 
several years to determine the characteristics of effective schools, leaders, and teachers. This 
most recent meta-analysis examines findings from 27 studies conducted since 1970 that 
used rigorous, quantitative methods to study the influence of school district leaders on 
student achievement. Altogether, these studies involved 2,817 districts and the 
achievement scores of 3.4 million students, resulting in what McREL researchers believe to 
be the largest-ever quantitative examination of research on superintendents. The following 
four major findings emerged from the study. 

Finding 1: District-level leadership matters 

The McREL research team, led by McREL President and CEO Tim Waters and McREL 
Senior Fellow Robert J. Marzano, found a statistically significant relationship (a positive 
correlation of .24) between district leadership and student achievement. 

Finding 2: Effective superintendents focus their efforts on creating goal- 
oriented districts 

McREL researchers also identified five district-level leadership responsibilities that have a 
statistically significant correlation with average student academic achievement. All five of 
these responsibilities relate to setting and keeping districts focused on teaching and 
learning goals. 

1. Collaborative goal-setting 

Researchers found that effective superintendents include all relevant stakeholders, 
including central office staff, building-level administrators, and board members, in 
establishing goals for their districts. 
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2. Non-negotiable goals for achievement and instruction 

Effective superintendents ensure that the collaborative goal-setting process results in non- 
negotiable goals (i.e., goals that all staff members must act upon) in at least two areas: 
student achievement and classroom instruction. Effective superintendents set specific 
achievement targets for schools and students and then ensure the consistent use of 
research-based instructional strategies in all classrooms to reach those targets. 

3. Board alignment and support of district goals 

In districts with higher levels of student achievement, the local board of education is 
aligned with and supportive of the non-negotiable goals for achievement and instruction. 
They ensure these goals remain the primary focus of the district’s efforts and that no other 
initiatives detract attention or resources from accomplishing these goals. 

4. Monitoring goals for achievement and instruction 

Effective superintendents continually monitor district progress toward achievement and 
instructional goals to ensure that these goals remain the driving force behind a district’s 
actions. 

5. Use of resources to support achievement and instruction goals 

Effective superintendents ensure that the necessary resources, including time, money, 
personnel, and materials, are allocated to accomplish the district’s goals. This can mean 
cutting back on or dropping initiatives that are not aligned with district goals for 
achievement and instruction. 

Finding 3: Superintendent tenure is positively correlated with student 
achievement 

McREL found two studies that looked specifically at the correlations between 
superintendent tenure and student achievement. The weighted average correlation in these 
two studies was a statistically significant .19, which suggests that length of superintendent 
tenure in a district positively correlates to student achievement. These positive effects 
appear to manifest themselves as early as two years into a superintendent’s tenure. 

A surprising & perplexing finding: "Defined autonomy" 

One set of findings from the meta-analysis that at first appears contradictory involves 
building-level autonomy within a district. One study reported that building autonomy has a 
positive correlation of .28 with average student achievement in the district, indicating that 
an increase in building autonomy is associated with an increase in student achievement. 
Interestingly, that same study reported that site-based management had a negative 
correlation with student achievement of (-) .16, indicating that an increase in site-based 
management is associated with a decrease in student achievement. Researchers concluded 
from this finding that effective superintendents may provide principals with “defined 
autonomy.” That is, they may set clear, non-negotiable goals for learning and instruction, 
yet provide school leadership teams with the responsibility and authority for determining 
how to meet those goals. 
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Background 

I n 1998, McREL began a series of meta-analytic studies that we view as third-generation 
effective schools research. The first generation of effective schools research, conducted 
from the late 1960s to the mid-1980s, produced the first set of “effective schools correlates” 
— school-level practices that researchers found more evident in schools with higher levels of 
student achievement than in schools with lower levels of student achievement, even when 
accounting for variances in student backgrounds and socioeconomic status. These 
correlates included practices such as the following. 

• Safe and orderly environment 

• Strong instructional leadership 

• High expectations for student achievement 

• Clear and focused mission 

• Time on task 

Findings from this first generation of research established the first empirical relationship 
between practices used in schools and student achievement. The general conclusion drawn 
from these studies was that what happens in schools matters. Differences in achievement 
among schools are not just a reflection of the characteristics of students who attend them, 
but also the efforts of professionals within those schools. 

As helpful as these findings were, the effective school correlates lacked sufficient specificity 
for practitioners to distinguish clearly and consistently between truly effective and 
ineffective practices. Nor did the first generation of effective schools research compute the 
strength of the relationships between identified practices and student achievement. The 
strength of these relationships have generally been reported as effect sizes. Although many 
types of effect sizes can be used to report the strength of relationships (see Lipsey & 

Wilson, 2001), school effectiveness research most often reports effect sizes as correlation 
coefficients. 

In the 1970s, 80s, and 90s, researchers continued to examine the relationship between 
classroom practices, school practices, and student achievement. As the findings from these 
studies began to accumulate, a body of research-based knowledge emerged, along with 
increasingly robust sets of data for secondary analysis. This body of knowledge and these 
data evolved into the second generation of effective schools research. In this generation, 
researchers were able to more explicitly describe effective practices and compute the effect 
sizes, or strength of relationship, between specific practices and student achievement. 

The new, third generation of effective schools research translates well-defined, effective 
classroom, school, and leadership practices into specific actions and behaviors. These 
actions and behaviors represent the basic procedural, or “how-to,” knowledge practitioners 
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need to translate research into practices that produce high levels of student achievement. 
McREL’s contributions to this third generation of effective schools research has been 
published as a series of “what works” books, including Classroom Instruction that Works 
(Marzano, Pickering, & Pollock, 2001), What Works in Schools (Marzano, 2003), Classroom 
Management that Works (Marzano, Marzano, & Pickering, 2003), and School Leadership that 
Works (Marzano, Waters, & McNulty, 2005). Each study in this series was built on earlier 
studies and helped establish the foundation for subsequent analyses. Similarly, McREL’s 
most recent analysis of the effects of superintendent leadership on student achievement 
incorporates aspects of each of the previous “what works” studies — most notably the 
findings from the meta-analysis of research on school leadership, reported in the book, 
School Leadership the Works. 

In School Leadership that Works, we answered four important questions about school-level 
leadership. 

1. Does principal leadership have an effect on average student achievement in 
school? 

2. Are there specific leadership responsibilities that, when fulfilled skillfully, 
correlate with student achievement? 

3. What practices do principals use to fulfill leadership responsibilities? 

4. What is the variation in the relationship between school leadership and student 
achievement? Stated differently, do behaviors associated with strong leadership 
always have a positive effect on student achievement? 

The answer to the first question is yes. Principal leadership does have discernable effects on 
student achievement. In fact, we found the correlation between school level leadership and 
average student achievement in schools to be .25. 

We answered the second question by identifying 21 school-level leadership responsibilities 
with statistically significant correlations to student achievement. These 21 responsibilities 
do not represent all of the important responsibilities principals are expected to fulfill. They 
do, however, represent leadership responsibilities that, when fulfilled skillfully, positively 
impact student achievement. Of the many important responsibilities principals are expected 
to fulfill, the 21 reported in School Leadership that Works are essential to producing higher 
levels of student achievement (Waters &. Grubb, 2005). 

We answered the third question by identifying 66 practices principals use to fulfill the 21 
responsibilities that positively influence student achievement. The 21 responsibilities are 
generalizations about what principals and other school-level leaders do that positively 
influence achievement. The 66 practices are more specific descriptions of what they are 
doing to fulfill these responsibilities. 
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In answering the fourth question, we found that behaviors associated with strong 
leadership at the school level do not always have a positive relationship with student 
achievement. That is, we found studies in which principals were rated as strong leaders, yet 
student achievement levels were low in their schools. 

We postulated at least two plausible factors that could explain this finding. The first is the 
focus of principal leadership. Even strong leaders need to focus their attention and their 
school’s efforts on practices that are likely to improve student achievement. 

The second factor is the “magnitude of the change” implied by the leader’s focus (for more 
discussion, see pp. 17-19). In addition to focusing their attention and improvement efforts 
on practices that are highly likely to improve achievement, principals must also skillfully 
adapt their leadership behaviors based on the “order of magnitude” of the change implied 
by this focus. Failing to focus on the “right” practices, and/ or failing to effectively manage 
the change implied by these practices, can produce what we have called the “differential 
impact of leadership”— leadership that on the surface appears strong, but does not 
positively influence student achievement. 

Our findings, conclusions, and recommendations regarding school-level leadership, first- 
order and second-order change, and managing second-order change are summarized in 
School Leadership that Works (2005). After completing this study of school-level leadership, 
we turned our attention to superintendent leadership. Using the same methods we 
employed in our study of principals, we sought answers to the following research questions 
regarding superintendent and district-level leadership. 

Research questions 

We asked the following basic research question for our meta-analysis of research on 
superintendents: 

• What is the strength of relationship between leadership at the district level and 
average student academic achievement in the district? 

In addition, we asked the following related research questions: 

• What specific district-level leadership responsibilities are related to student 
academic achievement? 

• What specific leadership practices are used to fulfill these responsibilities? 

• What is the variation in the relationship between district leadership and 
student achievement? Stated differently, do behaviors associated with strong 
leadership always have a positive effect on student achievement? 

This working paper reports our initial answers to these questions. A more detailed and 
technical accounting of our findings, conclusions, and recommendations will be reported 
in the forthcoming book Leadership at the Top (Marzano & Waters, in preparation). As in 
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each of the “what works” studies, we used meta-analyses to synthesize quantitative research 
studies. Although not part of our initial set of questions, we are able to answer a fifth 
question that we believe to be of interest to superintendents and local school board 
members, but is not specifically focused on superintendent responsibilities and practices: 

• Is there a relationship between length of superintendent service and student 
achievement? 

We think of the answer to this fifth question as a “bonus” finding that was not initially 
part of our inquiry. 

Key Finding: District leadership makes a difference 

The answers we found to these five questions affirm the long-held, but previously 
undocumented, belief that sound leadership at the district level adds value to an education 
system. However, these answers stand in stark contrast to the image of superintendents, 
school boards, and district office staff created by former Secretary of Education William 
Bennett, who characterized superintendents, district office staff, and school board 
members as part of the education “blob.” 

Bennett first coined the term the “blob” in his state of education speech in the spring of 
1987 ( Education Week, March 2, 1987). The “blob,” he argued, is made up of people in the 
education system who work outside of classrooms, soaking up resources and resisting 
reform without contributing to student achievement. He reiterated this assertion in The 
Educated Child when he and his co-authors wrote: 

The public school establishment is one of the most stubbornly intransigent forces on the 
planet. It is full of people and organizations dedicated to protecting established programs and 
keeping things just the way they are. Administrators talk of reform even as they are circling 
the wagons to fend off change, or preparing to outflank your innovation ... To understand 
many of the problems besetting U.S. schools, it is necessary to know something about the 
education establishment christened the “blob” by one of the authors (Bennett, Finn, & Cribb, 
1999, p. 628) 

Bennett, Finn, and Cribb include superintendents, district office staff, and local school 
board members as part of the “blob.” Certainly, one could find examples of local school 
district bureaucracies that stand in the way of efforts to improve student learning. Indeed, 
our research supports the assertion that not all superintendent behaviors produce a 
positive impact on student achievement. However, our research does not support Mr. 
Bennett’s broad-stroke condemnation of superintendents, district office staff, and school 
board members. To the contrary, our findings indicate that when district leaders effectively 
address specific responsibilities, they can have a profound, positive impact on student 
achievement in their districts. 




Methodology 

T he methodology used in our study of district-level leadership was meta-analysis. The 
specifics of meta-analysis are detailed in a number of works (see Lipsey & Wilson, 

2001; Cooper & Hedges, 1994). In brief, meta-analysis is a series of quantitative techniques 
for synthesizing research regarding a specific topic. In this case, that topic is school district 
leadership. 

The targeted sample for our meta-analysis was all available studies involving district 
leadership or variables related to district leadership in the United States from 1970 until 
2005 that possess the following characteristics: 

• Reported a correlation between district leadership or district leadership 
variables and student academic achievement or allow for the computing or 
estimating of a correlation, and 

• Used a standardized measure of student achievement or some index based on a 
standardized measure of student achievement. 

To identify potential studies that met these criteria, four databases were queried: ERIC, 
PsychINFO, Dissertation Abstracts, and the AERA online search services. Keywords 
employed in those searches included: superintendent leadership, district leadership, effective 
superintendents, and effective districts. In all, over 4,500 non-repeating titles were retrieved. Of 
those titles, abstracts revealed that over 200 retrievable documents appeared to meet the 
identified parameters. These documents were retrieved and examined. Of those, 27 met 
the identified criteria. The demographics for these 27 reports were as follows: 

• Number of districts involved: 2,714 

• Number of ratings of superintendent leadership: 4,434 

• Estimated number of student achievement scores: 3.4 million 

Although there was a good deal of variation in the methodologies employed, the majority 
of studies surveyed superintendents regarding their perceptions of district-level variables. In 
some cases, the superintendents’ perceptions were combined with those of other related 
constituents such as board members, school-level administrators, and teachers. This 
perceptual data was then correlated with average student academic achievement at the 
district level. 
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Findings 

T he key findings of die meta-analysis for the basic research question and related 
questions are described below. 

The impact of district leadership on student achievement 

As noted earlier, we set out to answer the following basic research question: 

What is the strength of relationship between leadership at the district level and average 
student academic achievement in the district ? 

Of the 27 reports examined in the meta-analysis, 14 (excluding statistical outliers) 
contained information about the relationship between overall district-level leadership and 
average student academic achievement in the district. These 14 reports included data from 
1,210 districts. The computed correlation between district leadership and student 
achievement was .24 (95% confidence interval: .19 to .30). The fact that the 95 percent 
confidence interval does not include 0 indicates that this correlation is significant at the 
.05 level. 

Correlations such as these can be interpreted in a variety of ways (for a review see Cohen, 
Cohen, West, &. Aiken, 2003). One of the most common interpretations is to examine the 
expected change in the dependent variable associated with a one standard deviation gain in 
the independent variable (Magnusson, 1966). In this case, the independent variable is 
district-level leadership and the dependent variable is average student achievement in the 
district. 

One way to interpret the Figure 1: Effect size of leadership on student achievement 
.24 correlation is to 
consider an average 
superintendent who is at 
the 50 th percentile in terms 
of his or her leadership 
abilities and leading a 
district where average 
student achievement is 
also at the 50 th percentile. 

Now, assume that the 
superintendent improves 

his or her leadership abilities by one standard deviation (in this case, rising to the 84 th 
percentile of all district leaders). Given the correlation between district leadership and 
student achievement of .24, we would predict that average student achievement in the 
district would increase by 9.5 percentile points. In other words, average student 
achievement in the district would rise to the 59.5 th percentile as shown in Figure. 1. 
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Curve A in Figure 1 depicts a district at the 50 th percentile in terms of average student 
achievement and average district-level leadership. Curve B depicts the expected average 
academic achievement of students in the same district after the district leadership has 
increased in quality by one standard deviation. Again, average student academic 
achievement increased from the 50 th percentile to the 59.5 th percentile — a gain of almost 
10 percentile points. 

This finding stands in sharp contrast to the notion that district administration is a part of 
an amorphous blob that soaks up valuable resources without adding value to a district’s 
instructional program. To the contrary, these findings suggest that when district leaders are 
carrying out their leadership responsibilities effectively, student achievement across the 
district is positively affected. 

District leadership responsibilities correlated with student achievement 

Our second research question sought to identify the specific leadership responsibilities that 
produce gains in student achievement: 

What specific district leadership responsibilities are related to student academic achievement ? 

In response to this question, we found five district-level leadership responsibilities with a 
statistically significant (p < .05) correlation with average student academic achievement. 
They are as follows: 

• The goal-setting process 

• Non-negotiable goals for achievement and instruction 

• Board alignment with and support of district goals 

• Monitoring the goals for achievement and instruction 

• Use of resources to support the goals for achievement and instruction 
We describe each of these responsibilities in more detail in the following sections. 
Collaborative goal-setting 

Effective superintendents include all relevant stakeholders, including central office staff, 
building-level administrators, and board members, in establishing non-negotiable goals for 
their districts. In particular, they ensure that building-level administrators throughout the 
district are heavily involved in the goal-setting process since these are the individuals who, 
for all practical purposes, will implement articulated goals in schools. Involving principals 
and school board members in the goal setting process does not imply that consensus must 
be reached among these stakeholders. However, it does imply that once stakeholders reach 
an acceptable level of agreement regarding district goals, all stakeholders agree to support 
the attainment of those goals. 
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Non- negotiable goals for achievement & instruction 

Effective superintendents ensure that the collaborative goal-setting process results in non- 
negotiable goals (i.e., goals that all staff members must act upon) in at least two areas: 
student achievement and classroom instruction. This means that the district sets specific 
achievement targets for the district as a whole, for individual schools, and for 
subpopulations of students within the district. Once agreed upon, the achievement goals 
are enacted in every school site. All staff members in each building are aware of the goals 
and an action plan is created for those goals. 

With respect to goals for classroom instruction, this responsibility does not mean that the 
district establishes a single instructional model that all teachers must employ. However, it 
does mean that the district adopts a broad but common framework for classroom 
instructional design and planning, common instructional language or vocabulary, and 
consistent use of research-based instructional strategies in each school. 

Another characteristic of this responsibility is that all principals support the goals explicitly 
and implicitly. Explicit support means that school leaders engage in the behaviors described 
above. Implicit support means that building level administrators do nothing to subvert the 
accomplishment of those goals such as criticizing district goals or subtly communicating 
that the goals the district has selected are inappropriate or unattainable. 

Board alignment with & support of district goals 

In districts with higher levels of student achievement, the local board of education is 
aligned with and supportive of the non-negotiable goals for achievement and instruction. 
The board ensures that these goals remain the top priorities in the district and that no 
other initiatives detract attention or resources from accomplishing these goals. Although 
other initiatives might be undertaken, none can detract attention or resources from these 
two primary goals. Indeed, publicly adopting broad five-year goals for achievement and 
instruction and consistently supporting these goals, both publicly and privately, are 
examples of board-level actions that we found to be positively correlated with student 
achievement. 

It is not unusual that individual board members pursue their own interests and 
expectations for the districts they are elected to serve. This finding suggests, however, that 
when individual board member interests and expectations distract from board-adopted 
achievement and instructional goals, they are not contributing to district success, but, in 
fact, may be working in opposition to that end. 

Monitoring achievement & instruction goals 

Effective superintendents continually monitor district progress toward achievement and 
instructional goals to ensure that these goals remain the driving force behind a district’s 
actions. If not monitored continually, district goals can become little more than pithy 
refrains that are spoken at district and school events and highlighted in written reports. 
Effective superintendents ensure that each school regularly examines the extent to which it 
is to meeting achievement targets. Discrepancies between articulated goals and current 
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practices are interpreted as a need to change or to redouble efforts to enhance student 
achievement. In short, each school uses the achievement goals as their primary indicator of 
their success. The same can be said for instructional goals. Any discrepancies between 
expected teacher behavior in classrooms as articulated by agreed-upon instructional models 
and observed teacher behavior are taken as a call for corrective action. 

Use of resources to support the goals for instruction and achievement 

Superintendents of high-performing districts ensure that the necessary resources, including 
time, money, personnel, and materials, are allocated to accomplish the district’s goals. This 
can mean cutting back on or dropping initiatives that are not aligned with district goals for 
achievement and instruction. Our analysis does not answer questions about the level of 
resources school districts must commit to supporting district achievement and 
instructional goals. However, it is clear from our analysis that a meaningful commitment of 
funding must be dedicated to professional development for teachers and principals. The 
professional development supported with this funding should be focused on building the 
requisite knowledge, skills, and competencies teachers and principals need to accomplish a 
district’s goals. Furthermore, as professional development resources are deployed at the 
school level, they must be utilized in ways that align schools with district goals. 

A surprising & perplexing finding: "Defined autonomy" 

One set of findings from the meta-analysis that at first appears contradictory involves 
building-level autonomy within a district. One study reported that building autonomy has a 
positive correlation of .28 with average student achievement in the district, indicating that 
an increase in building autonomy is associated with an increase in student achievement. 
Interestingly, that same study reported that site-based management had a negative 
correlation with student achievement of (-) .16, indicating that an increase in site-based 
management is associated with a decrease in student achievement. 

Other studies on site-based management reported slightly better results. However, the 
average correlation between site-based management and student achievement was (for all 
practical purposes) 0. This apparent contradiction begins to make sense, however, in light 
of the five district-level leadership responsibilities described above. 

How can we find “school autonomy” positively correlated with student achievement and 
site-based management exhibiting a negligible or negative correlation with achievement? 
This question might be answered in at least two of the earlier findings. The superintendent 
who implements inclusive goal-setting processes that result in board adopted “non- 
negotiable goals for achievement and instruction,” who assures that schools align their use 
of district resources for professional development with district goals, and who monitors 
and evaluates progress toward goal achievement, is fulfilling multiple responsibilities 
correlated with high levels of achievement. When this superintendent also encourages 
strong school-level leadership and encourages principals and others to assume 
responsibility for school success, he or she has fulfilled another responsibility; to establish a 
relationship with schools. This relationship is characterized by “defined autonomy,” which 
is the expectation and support to lead within the boundaries defined by the district goals. 
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Superintendents in districts large enough to employ assistant superintendents, directors, 
and other administrative staff members, will quickly recognize the implications of this 
finding for their district office staff. A shared understanding of and commitment to a 
relationship with schools of “defined autonomy” is critical. In most large districts, 
superintendents fulfill responsibilities for planning, goal adoption, board alignment and 
support, resource alignment, and monitoring primarily through the district office staff. 
When an understanding of “defined autonomy” is shared and honored by all district office 
personnel, district-level leadership contributes positively to student achievement. When the 
district office staff is unable or unwilling to support schools’ “defined autonomy,” they may 
very well resemble what William Bennett labeled the “blob.” 

The "bonus" finding 

Our meta-analysis produced one finding that initially was not a focus of the study, but 
emerged from the analysis of the reports. Two studies that we examined reported 
correlations between superintendent tenure and student academic achievement. The 
weighted average correlation (corrected for attenuation) from these two studies was .19 
significant at the .05 level. 

This finding is rather profound in light of the discussion in the introductory section 
regarding the alleged lack of impact on student achievement attributed to the blob. 
Specifically, this finding implies that the longevity of the superintendent has a positive 
effect on the average academic achievement of students in the district. These positive 
effects appear to manifest themselves as early as two years into a superintendent’s tenure. 

Practices used to fulfill leadership responsibilities 

We also set out to answer the following research question: 

What specific leadership practices are used to fulfill these responsibilities ? 

From the studies we analyzed, we were able to extract specific practices used by 
superintendents to fulfill the six responsibilities described in the previous sections. Figure 2 
on the following pages lists these practices along with their average correlations. 
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Figure 2: Leadership responsibilities and practices 



Superintendent 

responsibilities 


Avg 

r 


Practices used by superintendent & executive/ district office staff to 
fulfill superintendent responsibilities 


Goal-setting process 

The superintendent involves 
board members and principals 
in the process of setting goals. 


.24 


Developing a shared vision for the goal setting process 

Using the goal setting process to set goals developed jointly by board and 
administration 

Developing goals that are coherent and reflect attendant values which support 
involvement and quality in achievement rather than maintenance of the 
status quo 

Communicating expectations to central office staff and principals 


Non- negotiable goals 
for achievement & 
instruction 

Goals for student achievement 
and instructional program are 
adopted and are based on 
relevant research. 


.33 


Modeling understanding of instructional design 

Establishing clear priorities among the district’s instructional goals and 
objectives 

Adopting instructional methodologies that facilitate the efficient delivery of the 
districts curriculum 

Incorporating varied and diverse instructional methodologies that allow for a 
wide range of learning styles that exist in a multi-racial student population 

Adopting 5-year non-negotiable goals for achievement and instruction 

Ensuring that a preferred instructional program is adopted and implemented 


Board alignment with 
& support of district 
goals 

Board support for district goals 
for achievement and 
instruction is maintained. 


.29 


Establishing agreement with the board president on district goals 

Establishing agreement with the board president on type and nature of conflict 
in the district 

Along with the board president, remaining situationally aware, agreeing on the 
political climate of the school district 

Establishing agreement with the board president on the nature of 
teaching/learning strategies to be used in the district 

Providing professional development for board members 

Establishing agreement with the board president on the effectiveness of board 
training 


Monitoring goals for 
achievement & 
instruction 

The superintendent monitors 
and evaluates implementation 
of the district instructional 
program, impact of instruction 
on achievement, and impact of 
implementation on 
implemented. 


.27 


Using an instructional evaluation program that accurately monitors 
implementation of the district’s instructional program 

Monitoring student achievement through feedback from the instructional 
evaluation program 

Using a system to manage instructional change 

Annually evaluating principals 

Reporting student achievement data to the board on a regular basis 

Ensuring that the curricular needs of all student populations are met 

Observing classrooms during school visits 

Coordinating efforts of individuals and groups within the organization to 
increase reliability of the system, with adjustments by individuals to quickly 
respond to system failures 
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Superintendent 

responsibilities 


Avg 

r 


Practices used by superintendent & executive/ district office staff to 
fulfill superintendent responsibilities 


Use of resources to 
support the goals for 
achievement & 
instruction 

Resources are dedicated and 
used for professional 
development of teachers and 
principals to achieve district 
goals 


.26 


Adopting an instructional and resource management system supporting 
implementation of the district’s instructional philosophy 

Providing extensive teacher and principal staff development 

Training all instructional staff in a common but flexible instructional model 

Controlling resource allocation 

Providing access to professional growth opportunities through the design of a 
master plan to coordinate in-service activities of the district 



The surprising and perplexing finding 



Superintendent Avg Practices used by superintendent & executive/ district office staff to 

responsibilities r fulfill superintendent responsibilities 



Defined autonomy; 
superintendent 
relationship with 
schools 

The superintendent provides 
autonomy to principals to lead 
their schools, but expects 
alignment on district goals and 
use of resources for 
professional development. 



Developing a shared vision and understanding of “defined autonomy” 

.28 Using standards for content and instruction as basic design principles 
Committing the district and schools to continuous improvement 
Screening, interviewing, and selecting teachers along with principals 
Hiring experienced teachers 

Rewarding successful teachers and terminating the employment of unsuccessful 
teachers 

Establishing teacher evaluation as a priority for principals 
Ensuring that principals speak with teachers about results 

Establishing strong agreed-upon principles/values which direct actions of 
people 

Ensuring that schools have a clear mission focused on school performance 

Ensuring that school practices are characterized by opportunity for all students 
to learn 

Including socializing functions in district meetings 

Maintaining high expectations for school performance 

Expecting principals to fulfill instructional leadership responsibilities 

Directing personnel operations to assure a stable yet improving and well- 
balanced work force 

Ensuring that schools are characterized by an orderly climate 
Promoting innovation 

Developing principal awareness of district goals and actions directed at goal 
accomplishment 

Providing leadership of curriculum development 

Ensuring that homogeneous ability groupings within classrooms do not 
segregate students into racial or other inappropriate groups 

Applying district sanctions to students for unsatisfactory academic performance 

Rewarding students beyond standard honor rolls and recognition assemblies for 
exceptional performance 



Note: The r correlations reported in this table are derived from McREL’s meta-analysis of research on superintendent leadership. 
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The differential impact of leadership 

Finally, we sought to answer the following research question: 

What is the variation in the relationship between district leadership and student 
achievement ? Stated differently, do behaviors associated with strong leadership always have a 
positive effect on student achievement ? 

We already have reported the general effect of district -level leadership. The correlation of 
district-level leadership with student achievement is .24. This is the “average” effect of 
leadership. Although this is the average effect, we found a range of effects with correlations 
as high as .54 and as low as -.13. This finding answers the related research question — there 
is a great deal of variation in the strength of relationship between district leadership and 
student achievement. Stated differently, behaviors associated with leadership at the district 
level are not always associated with an increase in average student achievement. We call 
this the “differential impact” of leadership. 

There are many possible explanations for the differential impact of leadership. There are 
two, however, that we view as most plausible. They are derived from our study of school- 
level leadership (see Marzano, Waters, & McNulty, 2005). First, the effect of strong 
leadership could be mitigated if a superintendent is focused (or focuses the district) on 
goals that are not likely to affect student achievement. As we stated earlier, a 
superintendent can focus the attention and resources of the district on many goals. Not all 
of them have the potential to influence student achievement. By focusing a district on 
goals that are unlikely to impact achievement, a seemingly strong superintendent can have 
a minimal or even negative effect on student performance. 

The second explanation for the differential impact of district-level leadership is the order of 
magnitude of change implied by the planning process, district goals, and alignment of 
resources. Even when the superintendent focuses the district on goals with the potential to 
improve achievement, he or she must accurately estimate the order of magnitude of change 
these goals imply for stakeholders. In our earlier work, we describe the characteristics of 
change that will be perceived as either first-order or second-order based on the implications 
of change for stakeholders. The terms first-order and second-order have as much to do with 
the implications of change for individuals expected to implement or who are impacted by it 
as they do with the specific features of change initiatives. 

The theoretical literature on leadership and change asserts that not all change is of the 
same order of magnitude (Heifetz, 1994; Fullan, 1993; Beckard & Pritchard, 1992; 
Hesselbein & Johnston, 2002; Bridges, 1991; Rogers, 1995; Nadler, Shaw, &. Walton 
1994; Kanter, 1985). Some changes represent more significant implications for staff 
members, students, parents, and community members than others. We have used the 
terms first-order and second-order to distinguish between changes perceived as routine and 
those perceived as dramatic. Leading change theorists have used such terms as technical vs. 
adaptive, incremental vs. fundamental, and continuous vs. discontinuous to make this 
same distinction. 
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In our earlier work, we list perceived characteristics of change that will cause it to be viewed 
as either first-order or second-order based on its implications for stakeholders. Whether a 
change is perceived as first-order or second-order has as much to do with what it implies for 
the individuals expected to implement it or who are impacted by it as it does with the 
specific features of a change initiative. 

Few changes can be considered as either first-order or second-order for all stakeholders. 

The same change may be viewed by a majority of stakeholders as first-order while at the 
same time it is perceived as second-order by a minority of stakeholders. The reverse can be 
true as well. What determines whether stakeholders perceive a change to be first-order or 
second-order is their own knowledge, experience, values, and flexibility. Figure 3 lists 
perceived characteristics of change that will cause stakeholders to perceive it as first-order 
or second-order. 



Figure 3: Perceptions that can cause change to be viewed as first- or second-order 



First-order Change 


Second-order Change 


When a change is perceived as: 


When a change is perceived as: 


An extension of the past 


A break with the past 


Within existing paradigms 


Outside of existing paradigms 


Consistent with prevailing values and norms 


Conflicted with prevailing values and norms 


Implemented with existing knowledge & skills 


Requiring new knowledge & skills to 
implement 



An example of a change that most teachers might view as first-order is teaching the 
vocabulary students must understand to perform well in their school’s assessment and 
accountability program. Teaching vocabulary that appears in the essential curriculum and 
in assessment instruments makes sense to most teachers. It is consistent with their prior 
experience, an incremental step that builds on the existing knowledge of pedagogy, is 
consistent with their personal values and the perceived norms of their school and district. 

However, this is not true for all teachers. For some teachers, vocabulary instruction can be 
a second-order change. It is not consistent with their prior experience, conflicts with their 
personal values and the prevailing norms of their school or district, and requires them to 
gain new knowledge and skills. As a result, an effort to encourage direct teaching of 
vocabulary throughout the district is a change that can be a first-order change for most 
stakeholders but a second-order change for others. 

Consider a second example: a decision to implement a system of standards-based record 
keeping, grading, and reporting. In this case, teachers would be asked to base their 
assessment of student performance on the standards or benchmarks adopted for their 
grade level or course of study. Grades would be calculated based on students’ demonstrated 
learning at the end of a grading or assessment period, rather than averaging performance 
from the beginning to the end of the grading period. “Report cards” or other forms of 
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reporting to students and parents would reflect student performance using a rubric tied to 
each benchmark. Grades would be a product of how students performed against the 
criteria included in the rubric. 

In most schools and districts, this approach to grading would represent a second-order 
change for most stakeholders. However, this may not be true for everyone. Some teachers, 
principals, and community members might view this change as a logical next step to their 
work with standards and benchmarks, consistent with their personal values and school and 
district policy development. For these stakeholders, this change would be a first-order 
change. 

To avoid the “differential impact of leadership,” it is necessary for superintendents to 
understand and to estimate accurately the order of magnitude the district’s goals will imply 
for different stakeholders. 
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Conclusion: Dispelling the Myth of the Blob 



F or two decades, superintendents, district office personnel, and school board members 
have worked to overcome the image of the “blob” created by William Bennett. 
Undoubtedly, there are school district bureaucracies for which this label applies. However, 
we have found a substantial and positive relationship between district-level leadership and 
student achievement when the superintendent, district office staff, and school board 
members do the “right work” in the “right way.” These findings suggest that 
superintendents, district office staff, and school board members can contribute to school 
and student success when they are focused on fulfilling key leadership responsibilities and 
using the practices reported in this study. In short, these findings help to dispel the myth 
of the “blob” perpetuated by Bennett, Finn, and Cribb. 



In addition, the positive correlations that appear between the length of superintendent 
service and student achievement confirms the value of leadership stability. Superintendents 
should note the importance of remaining in a district long enough to see the positive 
impact of their leadership on student learning and achievement. Of equal significance is 
the implication of this finding for school boards as they frequently determine the length of 
superintendent tenure in their districts. In his book Crash Course (2005), Chris Whittle 
contrasts CEO stability in major corporations with superintendent stability in large urban 
school districts (see Figures 4 and 5). 



Figure 4: Superintendent stability in selected urban districts 



City 


Number of superintendents in past 20 years 


Avg. tenure in years 1 


Kansas City 


14 


1.4 


Washington, D.C. 


9 


2.2 


New York City 


8 


2.5 


Figure 5: CEO stability 


in selected corporations 


Company 


Number of CEOs in the past 20 years 


Avg. tenure in years 


General Electric 


2 


11 


Federal Express 


1 


35 


Microsoft* 


1 


30 


Dell** 


1 


21 



* Bill Gates stepped down as CEO of Microsoft in 2000. Steve Ballmer now serves as Microsoft’s CEO. 
** Michael Dell stepped down as Dell’s CEO in 2004. Kevin Rollins now serves as Dell’s CEO. 



(Crash Course, P. 47) 
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The corporations listed in Figure 5 are generally acknowledged as among the most 
successful in the world. Whittle asserts that CEO stability accounts for a large percentage 
of their success. He also argues that the instability of superintendent leadership reflected in 
the school districts listed in Figure 4 accounts for much of the low student achievement 
found in too many school districts. If the stability of superintendents was to approximate 
the stability of CEO leadership, he claims, the performance of school districts would be 
enhanced. This obviously assumes the superintendent is focused on the “right” priorities 
and skillfully fulfilling his or her responsibilities. Our “bonus” finding of the relationship 
between superintendent stability and student achievement supports Whittle’s conclusion. 

School board members need to hire a superintendent who skillfully fulfills key leadership 
responsibilities. They need to support district goals for achievement and instruction. They 
need to support district- and school-level leadership in ways that enhance, rather than 
diminish, stability. When focused on effective classroom, school, and district practices, 
appropriate achievement and instructional goals, and effective leadership responsibilities, it 
is clear that school district leadership matters. Under these conditions, rather than be part 
of the “blob,” superintendents, district office staff, and school boards can be part of the 
solution. 
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